th-1cc9fa: model-output ceiling clamp + raise starvation defaults (.NET parity)#196
Open
brentrager wants to merge 1 commit into
Open
th-1cc9fa: model-output ceiling clamp + raise starvation defaults (.NET parity)#196brentrager wants to merge 1 commit into
brentrager wants to merge 1 commit into
Conversation
…ET server parity) Threads a per-turn output budget + the resolved model's hard output ceiling into the engine (which clamps max_tokens to min(budget, ceiling); EPIC th-1cc9fa), and raises the starvation-prone defaults that make reasoning models return empty. - ModelInfo: parse/fetch the model's max_output_tokens from the gateway's /model/info (C# analog of the Rust admin::model_output_ceiling + map_model_info), best-effort — any gateway error / unknown model / missing ceiling => null => unclamped. - TurnLimits: DI carrier with the raised defaults (max_tokens 512->8192, iterations 6->20) + the model ceiling; threaded through FrameDispatcher -> TurnRunner -> AgentOptions. - Program.cs: resolve the ceiling once at boot (single-model server) and register TurnLimits; SMOOTH_MAX_TOKENS / SMOOTH_MAX_ITERATIONS override. Bumps the core NuGet pin 1.3.0 -> 1.5.0 for the new AgentOptions clamp API. The clamp activates once core 1.5.0 publishes; the raised defaults are live now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
.NET server parity for the model-output-token ceiling clamp + starvation-default raise (siblings: TS #195, Python #194; Rust already merged).
ModelInfo— parse/fetch the model'smax_output_tokensfrom the gateway's/model/info(C# analog of the Rustadmin::model_output_ceiling+map_model_info). Best-effort: any gateway error / unknown model / missing ceiling ⇒null⇒ unclamped.TurnLimits— DI carrier (likeConfirmTools) with the raised defaults (max_tokens 512→8192, iterations 6→20) + the model ceiling. ThreadedFrameDispatcher → TurnRunner → AgentOptions(MaxOutputTokens/ModelMaxOutputTokens/MaxIterations); the engine clampsmax_tokenstomin(budget, ceiling).Program.cs— resolves the ceiling once at boot (single-model server, vs the Rust per-turn lazy cache) and registersTurnLimits;SMOOTH_MAX_TOKENS/SMOOTH_MAX_ITERATIONSoverride.Why
A budget
max_tokenscan exceed what a model can physically emit → a reasoning model burns the budget on reasoning and returns EMPTY, or the upstream 400s (e.g.groq-compoundcaps output at 8192). And 512/6 (chat-widget sizing) starve reasoning models outright.Files
dotnet/server/src/ModelInfo.cs(new),dotnet/server/src/TurnLimits.cs(new)dotnet/server/src/TurnRunner.cs,FrameDispatcher.cs,host/Program.cs,aspnetcore/SmoothOperatorWebSocketExtensions.csdotnet/server/tests/MaxTokensLimitsTests.cs(new) — 9 tests: ceiling parse (happy/dropped/malformed), best-effort fetch (known/unknown/error), raised defaults, TurnRunner threads the budget.dotnet/server/src/SmooAI.SmoothOperator.Server.csproj— core pin 1.3.0 → 1.5.0.The core clamp API (
AgentOptions.MaxOutputTokens/ModelMaxOutputTokens) ships in smooth-operator-core#74 (NuGet 1.5.0). Until that publishes,dotnet restoreon the .NET CI lane can't resolve 1.5.0 → the .NET job will be red. Verified locally against a locally-packed 1.5.0: full.slnxbuilds (12 projects) and all tests pass (server 65, integration 25, host 2, postgres 33, client 31/1-skip). The raised defaults are live immediately; the ceiling clamp activates once core 1.5.0 publishes and the pin resolves — same ordering the Go/Python agents used.🤖 Generated with Claude Code